Estimating the Mean Cover Time of a Semi-markov Process via Simulation

Consider a semi-Markov process that, after entering state /, next goes to state j with probability P/ y , and given that the next state isy, the time until the transition from / to j occurs is a random variable with distribution F/j having mean m(i,j). Starting in state 0, suppose we are interested in estimating n = E[T], where T, called the cover time, is the time until all of the states 1,2,..., m have been visited. Let fi(i,j) denote the expected time, given that the process has just entered state /, until it enters state j , and suppose that we are able to compute all of the values of i*{i,j) for the pairs i,j of interest. In Section 2, we show how \i can be efficiently estimated by a simulation of the embedded Markov chain with transition probabilities P,j. We then consider the problem of using simulation to estimate E[Tn], the mean time until n of the states 1, . . . ,m have been visited, where 1 < n < m. In Section 3, we present an estimator of E[Tn] that is recommended when n is not too small. A different simulation estimator, which involves a conditional expectation and uses random hazards as control variates, and which is preferable to the estimator of Section 3 when n is small, is presented in Section 4.